Skip to content

odb: scan all sources' packfiles before loose objects - #975

Open
tyrielv wants to merge 1 commit into
microsoft:vfs-2.55.0from
tyrielv:tyrielv/checkout-perf-2.55-fix
Open

odb: scan all sources' packfiles before loose objects#975
tyrielv wants to merge 1 commit into
microsoft:vfs-2.55.0from
tyrielv:tyrielv/checkout-perf-2.55-fix

Conversation

@tyrielv

@tyrielv tyrielv commented Aug 7, 2026

Copy link
Copy Markdown

Problem

Since v2.55, git checkout (and any branch-changing operation) is
dramatically slower on large VFS for Git / Scalar enlistments. A
same-commit branch switch on a ~2.4M-entry index went from under a
second to ~36s. Fixes #974.

Root cause

The upstream per-source object database refactor (first released in
v2.54.0) changed object lookup from "scan all packfiles, then all loose
objects" to "per source: packed then loose." For an object that lives in
an alternate's packfile -- the normal arrangement for VFS for Git and
Scalar enlistments, where a shared object cache is mounted as an
alternate -- the primary source's loose object store is now consulted
first. That loose lookup is a filesystem stat(), and because callers
such as cache_tree_fully_valid() pass ODB_HAS_OBJECT_RECHECK_PACKED
(which clears OBJECT_INFO_QUICK) the cached-loose-index fast path is
skipped, so a real stat() runs for every object.

cache_tree_fully_valid() walks the whole cache tree and calls
odb_has_object() for every node -- ~380k objects on this index -- each
incurring a wasted stat(). Full analysis, the instrumented breakdown
(loose_lstats: 380,944, odb_misses: 0), and the alternatives
considered are in #974.

Fix

Preserve the refactor's per-source encapsulation but restore the old
ordering: when there is more than one source, scan the packfiles of
every source first, then consult each source's loose store. Single-source
repositories are unaffected.

before after
wasted loose stat()s ~380,944 0
cache_tree_fully_valid() ~32s ~2s
same-commit branch switch (wall) ~36s ~6-7s

Test

t5615 adds a regression test: an object stored as a packed delta in
an alternate and loose in the main object store. %(deltabase) proves
the read resolves to the alternate's packfile (nonzero base) rather than
the loose copy (zero oid). It fails without this change and passes with
it. t5613, t1006, and t0410 remain green.

Notes

This is a targeted mitigation for the microsoft/git fork so the
regression can be addressed quickly. The change keeps the per-source
encapsulation intact and only alters the search order when alternates are
present. A broader upstream discussion of the ordering may be worthwhile
separately.

The object database refactor that introduced per-source object stores
(cb506a8 "odb: introduce \"files\" source" and the surrounding series,
first released in v2.54.0) changed how do_oid_object_info_extended()
searches for an object. It now iterates the sources and, within each
source, consults that source's packfiles and then its loose object
store before moving on to the next source.

Before that series the search consulted every packfile -- across the
primary object directory and all alternates -- before it looked at any
loose object. The refactor reversed that for the multi-source case:
for an object that lives in an alternate's packfile, the primary
source's loose object store is now consulted first. That loose lookup
is a filesystem stat(), and because callers such as
cache_tree_fully_valid() pass ODB_HAS_OBJECT_RECHECK_PACKED (which
clears OBJECT_INFO_QUICK) the cached-loose-index fast path is skipped
and a real stat() runs for every such object.

In a repository that keeps its objects in an alternate -- the common
arrangement for VFS for Git and Scalar enlistments, where a shared
object cache is mounted as an alternate -- this is a steep penalty.
cache_tree_fully_valid() walks the whole cache tree and calls
odb_has_object() for every node; on an enlistment with a ~2.4M-entry
index that is ~380k objects, each incurring a wasted stat() on the
primary loose store. A same-commit branch switch spent ~32s in
cache_tree_fully_valid() (two calls of ~16s), observed in the field as
a ~2x rise in median checkout duration after the client carrying the
refactor rolled out.

Restore the previous ordering without undoing the per-source
encapsulation: when there is more than one source, scan the packfiles
of every source first (OBJECT_INFO_SKIP_LOOSE) and only then consult
each source's loose store (OBJECT_INFO_SKIP_PACKED). The single-source
case is unchanged, so repositories without alternates keep the
existing path. With the fix the same branch switch spends ~2s in
cache_tree_fully_valid(), the ~380k wasted stat()s are gone, and
performance matches versions predating the refactor.

Assisted-by: Claude Opus 4.8
Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
@tyrielv
tyrielv marked this pull request as ready for review August 7, 2026 22:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v2.55 regression: cache_tree_fully_valid() makes branch-changing checkout ~40s on a large full index (even for same-commit switches)

1 participant